Scale Stain: Multi-Resolution Feature Enhancement in Pathology Visualization

Digital whole-slide images of pathological tissue samples have recently become feasible for use within routine diagnostic practice. These gigapixel sized images enable pathologists to perform reviews using computer workstations instead of microscopes. Existing workstations visualize scanned images by providing a zoomable image space that reproduces the capabilities of the microscope. This paper presents a novel visualization approach that enables filtering of the scale-space according to color preference. The visualization method reveals diagnostically important patterns that are otherwise not visible. The paper demonstrates how this approach has been implemented into a fully functional prototype that lets the user navigate the visualization parameter space in real time. The prototype was evaluated for two common clinical tasks with eight pathologists in a within-subjects study. The data reveal that task efficiency increased by 15% using the prototype, with maintained accuracy. By analyzing behavioral strategies, it was possible to conclude that efficiency gain was caused by a reduction of the panning needed to perform systematic search of the images. The prototype system was well received by the pathologists who did not detect any risks that would hinder use in clinical routine

Towards Grading Gleason Score using Generically Trained Deep convolutional Neural Networks

We developed an automatic algorithm with the purpose to assist pathologists to report Gleason score on malignant prostatic adenocarcinoma specimen. In order to detect and classify the cancerous tissue, a deep convolutional neural network that had been pre-trained on a large set of photographic images was used. A specific aim was to support intuitive interaction with the result, to let pathologists adjust and correct the output. Therefore, we have designed an algorithm that makes a spatial classification of the whole slide into the same growth patterns as pathologists do. The 22-layer network was cut at an earlier layer and the output from that layer was used to train both a random forest classifier and a support vector machines classifier. At a specific layer a small patch of the image was used to calculate a feature vector and an image is represented by a number of those vectors. We have classified both the individual patches and the entire images. The classification results were compared for different scales of the images and feature vectors from two different layers from the network. Testing was made on a dataset consisting of 213 images, all containing a single class, benign tissue or Gleason score 3-5. Using 10-fold cross validation the accuracy per patch was 81 %. For whole images, the accuracy was increased to 89 %

Predicting breast tumor proliferation from whole-slide images : the TUPAC16 challenge

Tumor proliferation is an important biomarker indicative of the prognosis of breast cancer patients. Assessment of tumor proliferation in a clinical setting is a highly subjective and labor-intensive task. Previous efforts to automate tumor proliferation assessment by image analysis only focused on mitosis detection in predefined tumor regions. However, in a real-world scenario, automatic mitosis detection should be performed in whole-slide images (WSIs) and an automatic method should be able to produce a tumor proliferation score given a WSI as input. To address this, we organized the TUmor Proliferation Assessment Challenge 2016 (TUPAC16) on prediction of tumor proliferation scores from WSIs. The challenge dataset consisted of 500 training and 321 testing breast cancer histopathology WSIs. In order to ensure fair and independent evaluation, only the ground truth for the training dataset was provided to the challenge participants. The first task of the challenge was to predict mitotic scores, i.e., to reproduce the manual method of assessing tumor proliferation by a pathologist. The second task was to predict the gene expression based PAM50 proliferation scores from the WSI. The best performing automatic method for the first task achieved a quadratic-weighted Cohen's kappa score of κ = 0.567, 95% CI [0.464, 0.671] between the predicted scores and the ground truth. For the second task, the predictions of the top method had a Spearman's correlation coefficient of r = 0.617, 95% CI [0.581 0.651] with the ground truth. This was the first comparison study that investigated tumor proliferation assessment from WSIs. The achieved results are promising given the difficulty of the tasks and weakly-labeled nature of the ground truth. However, further research is needed to improve the practical utility of image analysis methods for this task

Diagnostic Review with Digital Pathology: Design of digitals tools for routine diagnostic use

Digital pathology is a novel technology currently being implemented world wide. Thisthesis summarizes four years of HCI and visualization research and provides an overallunderstanding of designing workstation software for pathologists. A human-centereddesign approach has been used to create a number of design interventions.The thesis covers three main areas of inquiry: Understanding pathologists’ problemsolving processes during diagnostic review, how to build different digital tools to supportthose processes, and how to incorporate digital image analysis algorithms when buildingthese tools.The thesis consist of a kappa that provides background and context, to the remainingappended papers. The papers describe studies covering, pathologists’ navigationstrategies in gigapixel sized images, the usability of different input devices and structuredreporting interfaces, how principles from volume rendering can be used for multi-scaleimages, and how make to use of machine learning algorithms to support pathologists’diagnostic processes.Together, these design projects show how digital pathology images can be usedto create tools to make pathologists more productive. This will make it possible forpathology laboratories to replace their diagonstic workflow using glass slides, with aworkflow based on digital images

Diagnostic Review with Digital Pathology: Design of digitals tools for routine diagnostic use

Digital pathology is a novel technology currently being implemented world wide. Thisthesis summarizes four years of HCI and visualization research and provides an overallunderstanding of designing workstation software for pathologists. A human-centereddesign approach has been used to create a number of design interventions.The thesis covers three main areas of inquiry: Understanding pathologists’ problemsolving processes during diagnostic review, how to build different digital tools to supportthose processes, and how to incorporate digital image analysis algorithms when buildingthese tools.The thesis consist of a kappa that provides background and context, to the remainingappended papers. The papers describe studies covering, pathologists’ navigationstrategies in gigapixel sized images, the usability of different input devices and structuredreporting interfaces, how principles from volume rendering can be used for multi-scaleimages, and how make to use of machine learning algorithms to support pathologists’diagnostic processes.Together, these design projects show how digital pathology images can be usedto create tools to make pathologists more productive. This will make it possible forpathology laboratories to replace their diagonstic workflow using glass slides, with aworkflow based on digital images

Designing a digital pathology workstation for routine practice

The role of the pathology lab is important in the future of cancer care. In orderto further personalize the care for cancer patients, more precise review of tumorspecimens is needed in order to guide clinicians between different treatmentstrategies.New digital imaging technologies is one promising possibility that might allowpathologists performing more and better work with the same amount ofresources. Early scanning systems and workstations have been shown to beinefficient and have not met the pathologists’ needs, who still perform most oftheir diagnostic work with mechanical microscopes.In this thesis, we analyze the pathologist\u27s work with early digital workstationsand present a set of new solutions in order to increase the performance of theinteraction with these systems.First, we review the implementation process of two current digital systems intwo pathology labs in Sweden (Paper I), followed by study of the navigationbehavior that is performed by pathologists when they explore large digital slidesof cancer specimens (Paper II).With a specific focus on design solutions that work within medical routinepractice, three different input devices for navigation in large images wascompared with pathologists as participants (Paper III), as well as a visualizationtechnique, inspired by semantic zoom in order to facilitate certain tasks forpathologists (Paper IV).The results provided in this thesis points towards the same conclusion that havemade in other domains: When good usability engineering is combined withtechnological advances, this can make novel technology become useful for real.For a Human-Computer Interaction (HCI) researcher, the pathologist caserepresents an especially demanding use of zoomable user interfaces. This hasdriven us to enhance efficiency of such interfaces further in order for them tobecome useful. The research findings offered within this thesis are particularlyimportant to the field of digital pathology. However, our findings could alsohave a bearing on the design of zoomable user interfaces

Designing a digital pathology workstation for routine practice

The role of the pathology lab is important in the future of cancer care. In orderto further personalize the care for cancer patients, more precise review of tumorspecimens is needed in order to guide clinicians between different treatmentstrategies.New digital imaging technologies is one promising possibility that might allowpathologists performing more and better work with the same amount ofresources. Early scanning systems and workstations have been shown to beinefficient and have not met the pathologists’ needs, who still perform most oftheir diagnostic work with mechanical microscopes.In this thesis, we analyze the pathologist\u27s work with early digital workstationsand present a set of new solutions in order to increase the performance of theinteraction with these systems.First, we review the implementation process of two current digital systems intwo pathology labs in Sweden (Paper I), followed by study of the navigationbehavior that is performed by pathologists when they explore large digital slidesof cancer specimens (Paper II).With a specific focus on design solutions that work within medical routinepractice, three different input devices for navigation in large images wascompared with pathologists as participants (Paper III), as well as a visualizationtechnique, inspired by semantic zoom in order to facilitate certain tasks forpathologists (Paper IV).The results provided in this thesis points towards the same conclusion that havemade in other domains: When good usability engineering is combined withtechnological advances, this can make novel technology become useful for real.For a Human-Computer Interaction (HCI) researcher, the pathologist caserepresents an especially demanding use of zoomable user interfaces. This hasdriven us to enhance efficiency of such interfaces further in order for them tobecome useful. The research findings offered within this thesis are particularlyimportant to the field of digital pathology. However, our findings could alsohave a bearing on the design of zoomable user interfaces

Improving the creation and reporting of structured findings during digital pathology review

Background: Today, pathology reporting consists of many separate tasks, carried out by multiple people. Common tasks include dictation during case review, transcription, verification of the transcription, report distribution, and report the key findings to follow-up registries. Introduction of digital workstations makes it possible to remove some of these tasks and simplify others. This study describes the work presented at the Nordic Symposium on Digital Pathology 2015, in Linköping, Sweden. Methods: We explored the possibility to have a digital tool that simplifies image review by assisting note-taking, and with minimal extra effort, populates a structured report. Thus, our prototype sees reporting as an activity interleaved with image review rather than a separate final step. We created an interface to collect, sort, and display findings for the most common reporting needs, such as tumor size, grading, and scoring. Results: The interface was designed to reduce the need to retain partial findings in the head or on paper, while at the same time be structured enough to support automatic extraction of key findings for follow-up registry reporting. The final prototype was evaluated with two pathologists, diagnosing complicated partial mastectomy cases. The pathologists experienced that the prototype aided them during the review and that it created a better overall workflow. Conclusions: These results show that it is feasible to simplify the reporting tasks in a way that is not distracting, while at the same time being able to automatically extract the key findings. This simplification is possible due to the realization that the structured format needed for automatic extraction of data can be used to offload the pathologists′ working memory during the diagnostic review

Combatting out-of-distribution errors using model-agnostic meta-learning for digital pathology

Clinical deployment of systems based on deep neural networks is hampered by sensitivity to domain shift, caused by e.g. new scanners or rare events, factors usually overcome by human supervision. We suggest a correct-then-predict approach, where the user labels a few samples of the new data for each slide, which is used to update the network. This few-shot meta-learning method is based on Model-Agnostic Meta-Learning (MAML), with the goal of training to adapt quickly to new tasks. Here we adapt and apply the method to the histopathological setting by identifying a task as a whole-slide image with its corresponding classification problem. We evaluated the method on three datasets, while purposefully leaving out-of-distribution data out from the training data, such as whole-slide images from other centers, scanners or with different tumor classes. Our results show that MAML outperforms conventionally trained baseline networks on all our datasets in average accuracy per slide. Furthermore, MAML is useful as a robustness mechanism to out-of-distribution data. The model becomes less sensitive to differences between whole-slide images and is viable for clinical implementation when used with the correct-then-predict workflow. This offers a reduced need for data annotation when training networks, and a reduced risk of performance loss when domain shift data occurs after deployment.Copyright 2021 Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited. </p

Implementation of large‑scale routine diagnostics using whole slide imaging in Sweden: Digital pathology experiences 2006-2013

Recent technological advances have improved the whole slide imaging (WSI) scanner quality and reduced the cost of storage, thereby enabling the deployment of digital pathology for routine diagnostics. In this paper we present the experiences from two Swedish sites having deployed routine large-scale WSI for primary review. At Kalmar County Hospital, the digitization process started in 2006 to reduce the time spent at the microscope in order to improve the ergonomics. Since 2008, more than 500,000 glass slides have been scanned in the routine operations of Kalmar and the neighboring Link\uf6ping University Hospital. All glass slides are digitally scanned yet they are also physically delivered to the consulting pathologist who can choose to review the slides on screen, in the microscope, or both. The digital operations include regular remote case reporting by a few hospital pathologists, as well as around 150 cases per week where primary review is outsourced to a private clinic. To investigate how the pathologists choose to use the digital slides, a web-based questionnaire was designed and sent out to the pathologists in Kalmar and Link\uf6ping. The responses showed that almost all pathologists think that ergonomics have improved and that image quality was sufficient for most histopathologic diagnostic work. 38 \ub1 28% of the cases were diagnosed digitally, but the survey also revealed that the pathologists commonly switch back and forth between digital and conventional microscopy within the same case. The fact that two full-scale digital systems have been implemented and that a large portion of the primary reporting is voluntarily performed digitally shows that large-scale digitization is possible today